Interactive systems and methods for video compositing

ABSTRACT

Certain interactive role performance systems and methods are disclosed that allow users to select a role to play in a prerecorded video, such as a movie scene, and to replace an original actor of that role with the user&#39;s performance. In certain examples, the user selects a prerecorded scene from which a character has been removed and records his or her performance. The interactive role performance system can insert the user&#39;s performance in place of the original character, thereby creating the appearance that the user is interacting, such as in real-time, with the other characters in the movie scene. Moreover, systems and methods can further utilize background subtraction techniques to isolate the user image. In further examples, the recorded user performance is stored separately from the prerecorded video.

RELATED APPLICATIONS

This application claims the benefit of priority under 35 U.S.C. §119(e)of U.S. Provisional Patent Application No. 61/077,363, filed Jul. 1,2008, and entitled “Interactive Systems and Methods for VideoCompositing,” and U.S. Provisional Patent Application No. 61/144,383,filed Jan. 13, 2009, and entitled “Interactive Systems and Methods forVideo Compositing,” the entirety of each of which is hereby incorporatedherein by reference to be considered part of this specification.

The present application is also related to the following applicationsfiled on even date herewith, each of which is hereby incorporated hereinby reference in its entirety to be considered part of thisspecification: U.S. patent application Ser. No. 12/495,548, entitled“CONTENT PREPARATION SYSTEMS AND METHODS FOR INTERACTIVE VIDEO SYSTEMS”;and U.S. patent application Ser. No. 12/495,590, entitled “USERINTERFACE SYSTEMS AND METHODS FOR INTERACTIVE VIDEO SYSTEMS”.

BACKGROUND

1. Field

Embodiments of the invention generally relate to interactive systems andmethods for performing video compositing in an entertainmentenvironment.

2. Description of the Related Art

Interactive entertainment is a popular leisure activity for peopleacross the globe. One favorite activity for many is karaoke, whichtemporarily turns lay persons into “stars” as they sing the lyrics to afavorite song. Karaoke machines play the music of a selected song whilesimultaneously displaying the song lyrics to a user.

Another favorite leisure activity for millions is watching movies.Billions of dollars are spent each year on movie purchases and rentalsfor home use. Home movie watching, however, has predominantly been apassive activity, wherein there is little if any viewer interaction.Furthermore, although one may watch the same movie repeatedly, each timethe same characters appear and recite the same lines and perform thesame actions.

SUMMARY

In view of the foregoing, a need exists for interactive systems andmethods for video compositing allowing a more seamless integration withexisting video scenes. Moreover, there is a need for systems and methodsthat can provide a real-time output of a combined video. Further, thereis a need for systems and methods that users can operate with littleskill or experience. Finally, there is a need for systems and methodsthat can generate media content, such as content wherein a character hasbeen removed, for interactive role performance systems.

In certain embodiments, an interactive role performance system allowsusers to select a role to play in a movie scene and replace the originalactor of that role with their own performance. Using the interactiverole performance system, if a participant wants to reenact scenes from afavorite movie, the participant can select a scene from that movie,record his or her own performance, and the system inserts thatperformance in place of the original character, creating the appearancethat the participant is interacting with the other characters in themovie scene. For example, if a participant wants to reenact a scene fromSTAR WARS, he can record his own performance as LUKE SKYWALKER and thatperformance is combined into the scene in place of the actor's (e.g.,Mark Hamill) performance.

In some embodiments, a content preparation system is used to generatethe scenes used by the interactive role performance system. Originalmedia content from a variety of sources, such as movies, television, andcommercials, can be used to provide participants with a wide variety ofscenes and roles. The content preparation system takes an original mediacontent, removes a character from the content, and recreates thebackground. By recreating the background after removing the character,the user is given greater freedom to perform as the user can performanywhere within the scene. For example, a scene from STAR WARS isgenerated by removing the LUKE SKYWALKER character from the scene, andrecreating the background behind LUKE SKYWALKER, leaving a clear,recreated background where the participant's performance can beinserted.

In certain embodiments, a method is disclosed for combining video imagesin an interactive video system. The method comprises receiving a firstvideo signal associated with prerecorded video content having multipleframes and receiving metadata associated with the first video signal,the metadata being indicative of selected ones of the multiple frames inwhich an original character of the prerecorded video content has beenremoved prior to receiving the first video signal. The method furtherincludes receiving a second video signal comprising second video contenthaving a background image and a video image of a participant playing arole of the original character in front of the background image andfiltering the second video signal to remove the background image tocreate a modified second video signal with the video image of theparticipant. The method also includes combining the first video signaland the modified second video signal to create combined video content,the combining comprising selectively inserting the video image of theparticipant in the selected ones of the multiple frames such that theparticipant appears in place of the original character by overlaying themodified second video signal on the first video signal such thatplacement of the video image of the participant is unrestricted withineach of the selected ones of the multiple frames.

In some embodiments, a system is disclosed for interactive videoentertainment. The system comprises a first database comprisingprerecorded video content having multiple frames and metadata associatedwith the prerecorded video content, the metadata identifying selectedones of the multiple frames in which an original character of theprerecorded video content has been removed. The system further comprisesa video input configured to receive a real-time video signal comprisinga video image of a participant and a background image and a processingmodule configured to execute on a computing device. The processingmodule is further configured to filter the real-time video signal toremove the background image to create a modified real-time video signal,and combine, based at least on the metadata, the prerecorded videocontent and the modified real-time video signal to selectively insertthe video image of the participant in the selected ones of the multipleframes such that the video image of the participant appears in place ofthe original character and such that the video image of the participantcan appear anywhere within each of the selected ones of the multipleframes.

In certain embodiments, a system is disclosed for combining videoimages. The system comprises: means for identifying selected ones ofmultiple frames of prerecorded video content in which an originalcharacter has been removed from the prerecorded video content; means forreceiving a second video signal comprising a video image of aparticipant and a background image; means for filtering the second videosignal to remove the background image and to create a modified secondvideo signal; and means for combining, based at least on the identifyingmeans, the prerecorded video content and the modified second videosignal to selectively insert the video image of the participant in theselected ones of the multiple frames such that the video image of theparticipant appears in place of the original character, wherein thecombining further comprises overlaying the modified second video signalon the prerecorded video content such that placement of the video imageof the participant is unrestricted within each of the selected ones ofthe multiple frames.

Furthermore, in certain embodiments, the systems and methods summarizedabove can advantageously be implemented using computer software. In oneembodiment, the system is implemented as a number of software modulesthat comprise computer executable code for performing the functionsdescribed herein. However, a skilled artisan will appreciate that anymodule that can be implemented using software to be executed on ageneral purpose computer can also be implemented using a differentcombination of hardware, software, and/or firmware.

For purposes of summarizing the disclosure, certain aspects, advantagesand novel features of the inventions have been described herein. It isto be understood that not necessarily all such advantages may beachieved in accordance with any particular embodiment of the invention.Thus, the invention may be embodied or carried out in a manner thatachieves or optimizes one advantage or group of advantages as taughtherein without necessarily achieving other advantages as may be taughtor suggested herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings, associated descriptions, and specific implementations areprovided to illustrate embodiments of the invention and not to limit thescope of the disclosure. In addition, methods and functions describedherein are not limited to any particular sequence, and the blocks orstates relating thereto can be performed in other sequences that areappropriate. For example, described blocks or states may be performed inan order other than that specifically disclosed, or multiple blocks orstates may be combined in a single block or state.

FIG. 1 illustrates an exemplary embodiment of an interactive roleperformance system according to certain embodiments of the invention.

FIG. 2 illustrates a flowchart of an exemplary embodiment of a videocompositing process according to certain embodiments of the invention.

FIG. 3 illustrates a flowchart of an exemplary embodiment of a mediacontent preparation process according to certain embodiments of theinvention.

FIG. 4A-4B illustrate alternative embodiments of the media contentpreparation process of FIG. 3.

FIGS. 5A-5D illustrate a frame of media content during various phases ofthe content preparation process in which a single actor is washed out ofthe scene.

FIGS. 6A-6B illustrate an exemplary matte layer created during the mediacontent preparation process of FIG. 3.

FIG. 7 illustrates an embodiment of a data flow diagram of aninteractive role performance system configured to operate with multipleplayers in different geographic locations.

FIG. 8 illustrates an embodiment of a wireframe for a video compositinginterface of the interactive role performance system of FIG. 1.

FIG. 9 illustrates an exemplary screen display of one embodiment of acascade interface for a video compositing interface.

FIG. 10 illustrates an exemplary screen display of one embodiment of themovement and selection process of the cascade interface of FIG. 9.

FIG. 11 illustrates an exemplary screen display of one embodiment of aperformance screen of a video compositing interface.

FIG. 12 illustrates an exemplary screen display of one embodiment of therole selection screen of a video compositing interface.

FIG. 13 illustrates an exemplary screen display of one embodiment of alarge screen view of a display window of a video compositing interface.

FIG. 14 illustrates an exemplary screen display of one embodiment of ascript printing screen of a video compositing interface.

FIG. 15 illustrates an exemplary screen display of one embodiment of thecamera setup screen of a video compositing interface.

FIG. 16 illustrates an exemplary screen display of one embodiment of areference frame setup screen of a video compositing interface.

FIG. 17 illustrates an exemplary screen display of one embodiment of anadd introduction screen of a video compositing interface.

FIGS. 18-20 illustrate exemplary screen displays of one embodiment ofthe setting screens of a video compositing interface.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Certain interactive role performance systems and methods are disclosedherein that allow users to select a role to play in a movie scene andreplace an original actor of that role with their own performance. Incertain embodiments, if a participant wants to reenact scenes from afavorite movie, the participant can select a scene from that movie,record his or her own performance, and the interactive role performancesystem inserts that performance in place of the original character,creating the appearance that the participant is interacting with theother characters in the movie scene.

In some embodiments, content preparation systems and methods areprovided that generate the scenes used by the interactive roleperformance system. Original media content from a variety of sources,such as movies, television, and commercials, can be used to provideparticipants with a wide variety of scenes and roles. In someembodiments, the content preparation system takes original mediacontent, removes a character from the content, and recreates thebackground. By recreating the background after removing the character,the participant is given greater freedom to perform as the user canperform anywhere within the scene.

The features of the systems and methods will now be described withreference to the drawings summarized above. Throughout the drawings,reference numbers are re-used to indicate correspondence betweenreferenced elements. The drawings, associated descriptions, and specificimplementation are provided to illustrate embodiments of the inventionand not to limit the scope of the disclosure.

For purposes of illustration, some embodiments will be described in thecontext of video formats and movie scenes. However, the presentdisclosure is not limited by the source of the media content, and othermedia content sources may be used, such as, for example, video games,animation, sports clips, newscasts, music videos, commercials,television, documentaries, combinations of the same or the like. Neitheris the present disclosure limited by the format of the media content,and other formats may be used, such as, for example, still images,computer generated graphics, posters, music, three-dimensional (3D)images, holograms, combinations of the above or the like. It is alsorecognized that in other embodiments, the systems and methods may beimplemented as a single module and/or implemented in conjunction with avariety of other modules and the like. Moreover, the specificimplementations described herein are set forth in order to illustrate,and not to limit, the invention.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that someembodiments include, while other embodiments do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

The terms “actor” or “character” as used herein are broad terms and areused in their ordinary sense and include, without limitation, anyreplaceable element in a media content, such as still or video content.For example, an “actor” or “character” can be a person (live oranimated), an animal, an avatar, a computer-generated character, a gamecharacter, a cartoon character, and/or a thing.

The terms “video,” “scene,” “clip,” “image,” and “content” are broadterms and are used in their ordinary sense and include, withoutlimitation, any type of media content. For example, media content caninclude pictures, videos, film, television, documentaries, commercials,sports, music, music videos, games, posters, original content,user-generated content, licensed content, royalty free content, anypre-existing moving image or graphic content, still images, digitalavatars, online content, combinations of the above, or the like. Themedia content may or may not include audio, dialogue, and/or effects.The media content can be in English or any other language.

The term “compositing” as used herein is a broad term and is used in itsordinary sense and includes, without limitation, the superimposing orcombining of multiple signals, such as, for example, video and/or audiosignals, to form a combined signal or display. Furthermore, compositingdoes not require two signals and/or video images to be stored as asingle signal, file and/or image. Rather, “compositing” can include thesimultaneous, or substantially simultaneous, playing of two or moresignals (for example, video files) such that the signals are output viaa single display or interface. The term “compositor” refers to anydevice or system, implemented in hardware, software, or firmware, or anycombination thereof, that performs in whole or in part a compositingfunction.

The term “real time” as used herein is a broad term and is used in itsordinary sense and includes, without limitation, a state or period oftime during which some event or response takes place. A real-time systemor application can produce a response to a particular stimulus or inputwithout intentional delay such that the response is generated during, orshortly thereafter, the receiving of the stimulus or input. For example,a device processing data in real time may process the data as it isreceived by the device.

Moreover, a real-time signal is one that is capable of being displayed,played back, or processed within a particular time after being receivedor captured by a particular device or system, wherein said particulartime can include non-intentional delay(s). In one embodiment, thisparticular time is on the order of one millisecond. In otherembodiments, the particular time may be more or less than onemillisecond. In yet other embodiments, “real time” refers to eventssimulated at a speed similar to the speed at which the events wouldoccur in real life.

The term “database” as used herein is a broad term and is used in itsordinary sense and includes without limitation any data source. Adatabase may be implemented using a relational database, such as DB2,Sybase, Oracle, CodeBase and MICROSOFT SQL SERVER as well as other typesof databases such as, for example, a flat file database, anentity-relationship database, and object-oriented database, and/or arecord-based database. A database may also be one or more files storedon a storage device, such as a hard drive or optical device.

The term “metadata” as used herein is a broad term and is used in itsordinary sense and includes without limitation any informationassociated with a media content. For example, the information cancomprise control data providing an interactive role performance systemwith directions on how to process the media content or the informationcan be descriptive information identifying the media content. Themetadata can comprise in and/or out points of characters, actions in theoriginal scene, audio levels, camera movement, switching, positions,zoom, pan, camera control signals, lighting information, color and hueinformation, titles, descriptions, category, tags, combinations of thesame or the like. Metadata can be recorded in a text document, database,eXtensible Markup Language (XML) file, and/or embedded within the washedor customized content.

FIG. 1 illustrates an exemplary embodiment of an interactive roleperformance system 100 according to certain embodiments of theinvention. In certain embodiments, the interactive role performancesystem 100 is configured to selectively insert an image of one or moreusers into prerecorded media, such as a movie. In some embodiments, theimage of the one or more users are recorded and/or inserted in realtime.

Referring to FIG. 1, the prerecorded content database 101 stores“washed” media content 104, or content wherein an actor or character hasbeen removed for replacement, and/or metadata 105. During contentdevelopment, a video source 102 receives, processes, and/or stores mediacontent received from a studio, comprising the source media files. Amedia content processing system 103 prepares or “washes” media contentclips of actors and/or objects and creates corresponding metadata 105for the washed media content 104. The completed washed content 104 isthen sent to the prerecorded content database 101.

From the content database 101, the washed media content 104 is availablefor use in the video composting process. The video recorder 110 capturesan image and/or video of the user. The feed from the video recorder 110is sent to the video compositor 120 and/or an optional user contentdatabase 115 for storage. The video compositor 120 accesses the washedcontent 104 stored on the prerecorded content database 101 and combinesthe washed content with the feed from the video recorder 110. The finalcombined output is shown on the display 125.

As illustrated, the interactive role performance system 100 comprises aprerecorded content database 101. The database 101 comprises data, videofiles, audio files, metadata and/or other information usable to controlthe video compositing process. For instance, the washed media content104 of the database 101 can comprise one or more video clips, such asmovie clips comprising video and/or audio content, usable in thebackground of a combined video image. In certain embodiments, the mediacontent 104 comprises washed content, as described in more detail below,wherein a character or other object has been removed from the mediacontent 104. In other embodiments, the media content 104 can compriseunaltered video scenes from a movie or other audiovisual work. Incertain embodiments, the video content comprises a QUICKTIME file, aMPEG file, a WMA file, a WMV file, a MP4 file, a MKV file, a JPEG file,and/or the like.

The database 101 can also comprise one or more matte files, as describedin more detail below, usable to overlay an inserted user image. Incertain embodiments, each matte file can be associated with one or moreparticular background clips. In other embodiments, the matte files canbe integrated with the background clips.

Referring to FIG. 1, the database 101 comprises metadata 105 usable tocontrol the selective combination of one or more character images withthe background media content 104 and/or matte files and/or to controlthe display of such images. For instance, such metadata 105 can comprisereference data files to control the display and/or removal of userimages, subtitle information, color/hue reference information (forexample, color or black and white output), information for moving mattefiles, resize data, movie poster-frame information for producingstill-frame movie posters, combinations of the same or the like. In someembodiments, the metadata 105 comprises descriptive informationassociated with the video content, such as actor information, key art,studio logos, titles, combinations of the same or the like.

The database 101 comprises any type of media device(s) and/or memorycapable of storing the above described information. For instance, thedatabase 101 can comprise one or more of the following: servers, harddrives, personal computers, DVDs, optical disks, flash memory, USBstorage devices, thumb drives, tapes, magnetic disks, combinations ofthe same or the like. Moreover, the database 101 can comprise multipledatabases located remote to each other.

In certain embodiments, the interactive role performance system 100further comprises a video source 102 and/or a media content processingsystem 103. These components can be used during content development toprocess original source content to produce washed content to be storedin the prerecorded content database. After washing the source content,some or all of the washed content can be, in certain embodiments, storedin the prerecorded content database as the washed media content 104.

In certain embodiments, the video source 102 can comprise one or morecomputers, workstations, servers, combinations of the same or the likefor processing and/or storing original source content. Source media fromstudios can be acquired in a variety of formats, such as digibeta tapes,digital files, DVDs, video tapes and/or the like. Source media from thestudios can be “ingested” by the video source 102 to create a copy foruse in the washing process and/or formatted into an uncompressed digitalfile. In some embodiments, the digital files are stored on a redundantarray of hard drives connected to the video source directly or through anetwork.

In certain embodiments, a playback machine, such as a digibeta playbackdeck, DVD player, video player, and/or the like may be used to play backthe source media with the video source ingesting the output. In certainembodiments, the video source 102 can further comprise a “helper” videocard with a Serial Digital Interface (SDI) and/or Audio EngineeringSociety digital audio (AES3) inputs/outputs to assist the video source102 in processing media content. In certain embodiments, the videosource stores digital files in a database. One or more hard drives canbe used for storing the master source. Additionally, master files can bebacked up on tape archival systems and/or stored on additional harddrives. Finished washed content can then be copied to one or moreprerecorded content database 101 as media content 104 for distributionto and/or use by participants.

In certain embodiments, the media content processing system 103processes the video source 102 digital files. The media contentprocessing system 103 can comprise, for example, computer workstationsequipped with speakers and/or headphones, video/audio editing software,timecode readers/generators, and/or house sync boxes. In someembodiments, the editing software comprises FINAL CUT PRO, PHOTOSHOP,AFTER EFFECTS, ADOBE AUDITION, SOUNDTRACK PRO, and/or the like.Operators can use the workstations and/or editing software to washindividual frames of scenes from the digital files of selected elements.Operators can further use the workstations and/or editing software torecreate the backgrounds behind the washed elements of the scenes. Insome embodiments, the media content processing system 103 furthercomprises a database and workflow manager to check the accuracy ofinformation, provide accessibility for production management, trackfinancial/royalty requirements, and/or provide archival security.

Referring to FIG. 1, the interactive role performance system 100 furthercomprises a video recorder 110, such as, for example, a digital videocamera, a web camera, a smart phone camera, combinations of the same orthe like, for obtaining one or more images to be inserted into thecombined video image. In certain embodiments, the video recorder 110obtains a real-time video image of a participant that is selectively“inserted” into a scene of the media content 104 from the prerecordedcontent database 101 to produce a real-time, interactive video image ata display.

In certain embodiments, the video recorder 110 further comprises, or isassociated with, a video content processor that modifies a video imageobtained through the recorder 110. For instance, such embodiments of thevideo recorder 110 can be used with a green screen, a blue screen,and/or other similar chroma-key equipment to prepare the obtained videoimage for compositing with the media content 104.

In other embodiments, video image captured through the video recorder110 can be digitally modified to remove certain portions of the capturedimage. For example, background subtraction techniques, as discussed inmore detail herein, can be used to isolate a foreground element, such asan image of the user. In certain embodiments, the video recorder 100further comprises a wired or wireless microphone, a remote control, atripod, combinations of the same or the like. Multiple video recorderslocated together or remotely from each other can be used to capturemultiple participants and/or multiple angles. In some embodiments, thevideo recorder 110 captures images in 3D and/or infrared formats.

The illustrated interactive role performance system 100 can alsocomprise an optional user content database 115. In certain embodiments,the user content database 115 stores video and/or audio data capturedthrough the video recorder 110. Such data can be stored directly to theuser content database 115 and/or can be further processed, as discussedherein, prior to such storage. Examples of such processing include, butare not limited to, removing, replacing, and/or enhancing videoelements, such as music, vocals, score, sound effects, special effects,combinations of the same or the like. The user content database 115, incertain embodiments, allows for the repeated and/or later playing of acombined video by storing a participant's “performance.” In otherembodiments, the user content database 115 stores avatar or othercomputer-generated images of a participant or other character for use inthe interactive role performance system 100.

In certain embodiments, the prerecorded content database 101 and/or theuser content database 115 can communicate with other components and/ormodules of the interactive role performance system 100 via a wired orwireless network such as, for example, a local area network, a wide areanetwork, the Internet, an intranet, a fiber optic network, combinationsof the same or the like.

As illustrated in FIG. 1, the interactive role performance system 100further comprises a video compositor module 120 configured to combinevideo images received from the prerecorded content database 101 withvideo images from the video recorder 110 and/or the user contentdatabase 115. In certain embodiments, the video compositor 120 comprisesat least a processor and a memory. Such memory can include, for example,SDRAM, EEPROM, flash, non-volatile memory, volatile memory, a harddrive, an optical drive, a combinations of the above or the like. Incertain embodiments, the video compositor 120 further comprises agraphics processing unit (GPU).

In certain embodiments, the video compositor module 120 advantageouslycombines and/or cause the display of multiple video images duringplayback without saving such images in a combined format. For instance,the media content 104 from the prerecorded content database 101 can becombined with the user video image from the video recorder 110 and/orthe user content database 115 to form a combined image without storingsuch content in a combined file. In other embodiments, the combinedimage is stored in a single file or location for later playback,comment, use and/or the like.

The video compositor module 120 is advantageously configured to causethe display 125 to output the combined video image. In certainembodiments, the display 125 can comprise a television, a monitor, aliquid crystal display (LCD), a cellular phone display, a computerdisplay, combinations of the same or the like. In certain furtherembodiments, the video compositor 120 can be integrated with the display125. Additional components can also be integrated together. For example,smart phone, PDA, or other mobile device could comprise the prerecordedcontent database 101, the video recorder 110 in the form of a camera,the video compositor 120 in the form of compositing software, and/or thedisplay 125. It is understood that components can be integrated in otherways, such as a camera with a memory for prerecorded content and aprocessor for running compositing software.

In certain embodiments, the prerecorded content database 101, videorecorder 110, user content database 115, video compositor 120, and/ordisplay 125 are used during video compositing when a user interacts withthe interactive role performance system 100. In certain embodiments, thecomponents used during video compositing can be provided to the userseparately from those components used in initially preparing anddeveloping the prerecorded content.

In certain embodiments, the content can be delivered to the user eitherthrough a physical medium or by online delivery. For instance, incertain embodiments, a user receives the video compositing elementswithout a prerecorded content database 101. Instead, the prerecordedcontent database 101 can be available online for the video compositor120 to access over a network and/or the Internet. In other embodiments,the user receives a prerecorded content database 101 containing alimited number of media content 104 files on a CD/DVD or other physicalmedium, with additional content available separately. For example,washed media content 104 can be stored on a central database from whichadditional content can be downloaded by the video compositor 120.

A variety of connection media may be used to link elements of theinteractive role performance system 100. In some embodiments, theelements are directly connected with fiber optic channels, Ethernet,and/or the like. Elements of the interactive role performance system 100can also be spread out in remote locations and connected through theInternet and/or a virtual private network. Elements of the interactiverole performance system 100 can also be wirelessly connected with otherelements. In some embodiments, elements, such as the video recorder 110,video compositor 120, and/or display 125 are connected directly withaudio and/or video cables and/or through a wireless connection, such asvia BLUETOOTH or other radio frequency communications.

Although the interactive compositing system 100 has been describedherein with reference to video technology, it will be understood fromthe disclosure herein that other types of media can be used by thesystem 100. For instance, such media can comprise video games,animation, still pictures, posters, combinations of the same or thelike.

FIG. 2 illustrates a flowchart of an exemplary embodiment of a videocompositing process 200, according to certain embodiments of theinvention. In certain embodiments, the video compositing process 200 isexecuted by the interactive role performance systems described herein toperform video compositing using washed content. For exemplary purposes,the video compositing process 200 is described hereinafter withreference to the components of the interactive role performance system100 of FIG. 1.

The video compositing process 200 begins with Block 205, during whichthe user of the interactive role performance system 100 can select anavailable washed scene from the media content 104 stored on theprerecorded content database 101. In certain embodiments, the userselection is made using some form of input device, such as a remotecontrol, a mouse, a keyboard, a touch screen, a keypad and/or the like.In certain embodiments, the video compositor 120 communicates with theprerecorded content database 101 to receive the washed media content104. The interactive role performance system 100 can further provide theuser or participant with a script of the lines or dialogue in a scene.In some embodiments, the interactive role performance system 100 furthercomprises a printer and provides the user with a printed script.

At Block 210, the scene selected by the user begins playing. In certainembodiments, the video compositor 120 displays the scene on the display125 with directions, prompts or instructions to the user for properlyacting out the scene. For instance, the directions can be text promptsand/or an outline on the display 125 directing the user to assume aspecified stance, position himself or herself in a specified location,and/or face a specified direction. Directions can also comprise of linesof dialogue that the user repeats, visual prompts on the screen, and/orvoice directions of actions to be taken, such as the directions that areal director would give to an actor.

At Block 220, the participant acts out the scene selected at Block 205.In certain embodiments, the participant follows the directional promptsgiven at Block 210. The directional prompts can be interactive and/orcan be given out while the user is acting out the scene.

In some embodiments, a real-time feed of the participant is displayedwhile the participant acts out the scene. The real-time feed gives theparticipant feedback on what the participant's actions look like onscreen. In certain embodiments, the prerecorded washed scene is combinedwith the real-time feed to provide the participant with real-timefeedback on what the final scene will look like. In some embodiments,graphics are superimposed over the real-time feed of the user to provideclearer directions to the user. The graphics can range from text tocomputer-generated graphics. In some embodiments, directional promptscan consist of interactive mini-games, directing the user to punchblocks, hit balls, hula dance, hula hoop, and/or the like, wherein theparticipant can act out the scene based on the directional prompts.

At Block 230, the image of the participant is captured by the videorecorder 110. In certain embodiments, the participant acts out the scenein front of a green screen. However, in other embodiments, screens ofdifferent colors or no screen can be used. Various techniques can alsobe used to isolate the image of the participant from the background. Forexample, chroma-key techniques can be used to separate the user from thebackground by the video compositor 120 and/or video recorder 110.

In some embodiments, a background processing technique is used to allowthe participant to act out the scene in front of any background, withthe background screen being optional. For example, the video compositor120 can use background subtraction, where a previously recordedreference image of the background is compared to the captured videoimage to identify a new element in the captured image, thereby isolatingthe image of the user. The new element is then identified as theforeground and/or is separated out from the background for insertioninto the media content 104 scene.

In certain embodiments, the user generated content captured by the videorecorded 110 is stored on the user content database 115. The videorecorder 110 can also record sound with the video. Moreover, in someembodiments, pre-existing sound clips are used and/or no sound isrecorded.

In some embodiments, the metadata 105 associated with the washed mediacontent 104 directs the video recorder 110 to turn on or off at certaintimes. The metadata 105 can be information contained in an XML file thatcontrols the video recorder 110. For example, when the image of theparticipant is not currently being inserted in the scene, the videorecorder 110 can be powered off or otherwise temporarily disabled toprevent extraneous sound and/or video from being recorded. In certainembodiments, the sound and the video capture can be controlledindependently of each other, providing greater control over how soundand/or video is captured by the video recorder 110.

At Block 240, the image of the participant is inserted into the washedmedia content 104 either as a video or a still image. The videocompositor 120 receives the user image from the video recorder 110and/or from the user content database 115. Various insertion techniquescan be used by the video compositor 120 to insert the participant'simage into the washed scene. For example, the image of the participantcan be played concurrently with and overlaid over the washed content, orthe image can be incorporated into the washed content.

In some embodiments, the image insertion is unrestricted, with the imageof the participant being capable of appearing anywhere within the washedscene. In certain embodiments, the metadata 105 or scene informationdirects the video compositor 120 on where the user image is to beinserted into the scene. This metadata 105 or scene information canfurther comprise display and removal points recording where areplaceable (or removed) actor appears in and/or exits the originalscene corresponding to the washed scene. In some embodiments, thedisplay and removal points comprise of the beginning and end frames ofthe scenes having the replaceable actor.

In certain embodiments, the metadata 105 also controls the insertion ofaudio, such as speech from the participant, into the washed content. Insome embodiments, the washed content also contains mattes, whichdetermine whether elements should appear in front of the inserted userimage. In some embodiments, the participant is inserted as an extra oradditional actor in a scene, without replacing an existing character.

In certain embodiments, the processing is offloaded to the GPU in thevideo compositor 120 to reduce load on a processor of the videocompositor 120.

In certain embodiments, the video compositing process 200 can comprise amulti-pass processing of the media content. For example, a two-passprocess can be used in which a first pass determines which pixels of theuser-generated content should be designated as transparent or opaque. Inparticular, the elements that are to be added to or combined with thewashed content 104 (e.g., the user image) are composed of opaque pixels.In certain embodiments, these pixel values are identified throughbackground subtraction processes described in more detail herein. In asecond pass, the user content is then inserted into the washed contentscene.

In some embodiments, further processing can be performed that blendsuser-generated content more cleanly with the washed media content 104.For example, a border of several pixels around the inserted content canbe blended into the washed content by applying a gradient of opaquenessto the border to create a more seamless integration with the washedcontent. Additional processing can be applied to the combined video toimprove the image. For example, pixel sampling can be conducted todetermine and correct the green levels in the image. Shadows, outlines,and/or color correction can also be applied to the combined video.

The following annotated source code illustrates one embodiment of thebackground subtracting program used to process the user-generated image.

// All values are normalized between 0.0 and 1.0. float4 main( float2theTextel : TEXCOORD0, float4 theColor : COLOR0 ) : COLOR0 { // grab theRGBA pixel color of the source and the background float4 aSrcColor =tex2D(srcTex, theTextel); float4 aBackColor = tex2D(tex1, theTextel); //convert the RGB values to HSV color space (hue, saturation, value)float3 aSrcHSV = RGBtoHSV((float3)aSrcColor); float3 aBackHSV =RGBtoHSV((float3)aBackColor); float3 aRBBDiff, aHSVDiff; float aMax; //find the difference in each RGB color channel aRBBDiff.r =abs(aSrcColor.r − aBackColor.r); aRBBDiff.g = abs(aSrcColor.g −aBackColor.g); aRBBDiff.b = abs(aSrcColor.b − aBackColor.b); // find thegreatest difference of all RGB color channels aMax = max(max(aRBBDiff.r,aRBBDiff.g), aRBBDiff.b); // find the difference in each HSV colorchannel aHSVDiff[0] = abs(aSrcHSV[0] − aBackHSV[0]); aHSVDiff[1] =abs(aSrcHSV[1] − aBackHSV[1]); aHSVDiff[2] = abs(aSrcHSV[2] −aBackHSV[2]); // the next lines return an opaque color value for thesource pixel if it matches one of the conditional criteria below //determine if the hue values differs from the threshold if (aHSVDiff[0] >0.075) return float4(aSrcColor.r, aSrcColor.g, aSrcColor.b, 1.0); //determine if the red values differs from the threshold if (aRBBDiff.r >0.25 && aMax == aRBBDiff.r) return float4(aSrcColor.r, aSrcColor.g,aSrcColor.b, 1.0); // determine if the green values differs from thethreshold if (aRBBDiff.g > 0.20 && aMax == aRBBDiff.g) returnfloat4(aSrcColor.r, aSrcColor.g, aSrcColor.b, 1.0); // determine if theblue values differs from the threshold if (aRBBDiff.b > 0.18 && aMax ==aRBBDiff.b) return float4(aSrcColor.r, aSrcColor.g, aSrcColor.b, 1.0);// if no value is determined to be opaque then set it to transparent.the default return float4(aSrcColor.r, aSrcColor.g, aSrcColor.b, 0.0); }

The exemplary program disclosed above takes the inputs of two images: asource image (e.g., a video image including the user) and a referencebackground image (e.g., without the image of the user) to be removedfrom each frame of the source image. For instance, the interactive roleperformance system 100 can record the reference background image afterthe user steps out of the video recorder 110.

In certain embodiments, the program processes one pixel or textel at atime and returns a value that is fully transparent or opaque based oncolor channel differences between the source image and the referencebackground. In the described source code, the portions of the sourceimage matching portions of the reference background are set transparent,while the portions of the source image that are different from thereference background are kept opaque. Collectively, the pixels in aframe that are opaque determine the actor and/or object that was addedto the scene after the reference background image was captured.

In certain embodiments, certain established threshold values determineif a particular pixel is to be designated as opaque or not. Moreover,the threshold value can compensate for small variations in the sourceimage and the reference background image due to inconsistencies in howthe source and background images were recorded. For example, if thelighting is inconsistent during the image acquisition process, thebackground from the source and background reference may not be recordedidentically. By using a threshold value, small differences between thebackgrounds are ignored and the background of the source image is settransparent. Depending on the level of inconsistencies between thesource image and the reference background, the threshold level could beset as a higher value to compensate for greater differences. In someembodiments, the threshold value can be a set value, set by the user, oradaptively set by the role performance system 100.

As described, the program further determines the RGB pixel color of thesource and reference background images and then converts the RGB valuesto HSV color space. The program then determines the difference in eachRGB and HSV color channel, wherein RGB color channels are red, green,and blue and HSV color channels are hue, saturation, and value. Theprogram determines the greatest difference of all RGB color channels.The RGB and HSV difference is measured against the threshold value todetermine if the pixel should be set as opaque. Otherwise, the pixel isset to transparent.

In certain further embodiments, an additional reference frame is takenof the participant within the background. By using a participantreference frame and the reference background image, the output qualityof the background processing can be checked using the two referenceframes. For example, a background subtraction process can be performedon the participant reference frame instead of the entire user content.The process outputs an isolated image of the participant which, incertain embodiments, is representative of the quality of the output fromprocessing the entire user content. Using the participant referenceframe allows the output quality of the background processing to betested with smaller files and less processing.

In certain embodiments, video images can be processed by the videocompositor 120 and/or by a remote transcoding server. For example, thecombined video can be encoded by the video compositor 120 while the usercontent is captured. In some embodiments, user content is captured inraw format and encoded at a later time by a transcoding server locatedremote to the video recorder 110. After capture, the user content can beuploaded to the transcoding server. In some embodiments, the transcodingserver is a virtual server running on a cluster of servers. As such, thetranscoding server has significantly more processing power than thevideo compositor 120. The additional processing power allows thetranscoding server to engage in additional processing passes over theuser content and the washed content to provide a higher quality video.In some embodiments, the transcoding server transcodes the content intoa flash video file, MPEG, JPEG, and/or other file type for viewing at adisplay 125 and/or for submission to a content sharing website. Thetranscoding server, in certain embodiments, can further apply awatermark to the user-generated content and/or the displayed combinedcontent for copy control purposes.

At Block 250, the combined video is shown on the display 125. In certainembodiments, the combined video is shown in real-time with respect tothe capturing of the user image by the video recorder 110. Customfilters can further be applied during playback to improve the displayedimage. In some embodiments, the combined video is saved for laterplayback and/or displayed at a later time. The video can also bedisplayed remotely at a location different from the user. It isunderstood that the combined video is not necessarily stored in a singlefile. The combined video can exist as separate files that are overlaidonto each other, played back, and/or synchronized to generate a combinedimage. The combined video can comprise of a matte file, a subtitle file,a washed content file, and/or a user content file.

In some embodiments, the combined video or elements of the combinedvideo, such as the user content file, are uploaded to a website forsearching, sharing, and/or viewing combined video content. For example,the user content can be sent to the website and played back along with acorresponding washed content stored on the website to generate thecombined video. In addition, the participant can create an introductionfor the combined video using the interactive role performance system 100using a process similar to that used to create the combined video.

The media content preparation process or “washing” process is aprocessing development for audio and/or video that increases the realismof the interactive experience. For example, in certain conventionalcompositing systems, a user playing the role of DARTH VADER in STAR WARSwould be positioned precisely in front of the villain before startingthe scene. During the scene, if the user were to move from side to side,the original DARTH VADER character would be visible behind the userimage, detracting from the supposed realism of the experience. However,the washing processes described herein advantageously removes theoriginal DARTH VADER character from the scene such that the participantimage need not be limited to a particular area and can be freely able tomove within the scene, thereby increasing the realism of the experience.

Likewise, the washing process can be applied to the audio of a scene. Incertain conventional compositing systems, data files accompanying theclip supply audio switch data in real time, turning various audio trackson or off in order to silence the replaceable character so the user canplay the part uninterrupted. In one embodiment of the washing process,the character's audio is filtered out of the original audio to create amodified audio used in the washed content, allowing the participantincreased freedom in the timing of his or her lines.

In certain embodiments, more content versatility is provided bydisclosed systems and methods that are able to simplify the compositingprocess by moving more of the content processing to the contentdevelopment phase, improving control over both audio and/or videomanipulation, and thereby improving the “replacement” effect. Forexample, with video, editing software can be used to entirely removeactors from a scene, allowing the washed content to be used as theprerecorded background into which the user is inserted and simplifyingthe user image insertion step during the compositing process.

FIG. 3 illustrates a flowchart of an exemplary embodiment of a mediacontent preparation process 300, used during content developmentaccording to certain embodiments of the invention. In certainembodiments, the process 300 is executed by embodiments of theinteractive role performance systems described herein, usually by thevideo source 102 and/or media content processing system 103. Forexemplary purposes, the media content preparation 300 is describedhereinafter with reference to the components of the interactive roleperformance system 100 of FIG. 1.

At Block 305, a scene is selected from media content stored on the videosource 102. In certain embodiments, a scene is identified by one or morewatchers viewing the entire source media content to select scenes thatcan viably be used as washed content. For instance, the one or morewatchers can log the start/end times of the scene. In some embodiments,media content is copied onto a network media server and then reviewed byone or more watchers.

In some embodiments, scenes are selected based on certain predeterminedcriteria and/or the ease with which the source content can be washed.Such selection criteria can comprise the duration of the scene, thevisibility of the primary actor, the immobility of the background, theminimal motion of the foreground, a clear view of the actors with littleor no blocking objects, and/or the consistency of the background. Incertain embodiments, scenes are generally avoided if the camera is inmotion, the background is in motion, there is a large amount offoreground action, there are many camera angles, scenes have lots ofactions, or scenes have lots of overlapping dialogue. In someembodiments, a media content clip, comprising a selected scene, iscaptured from the media content. In certain embodiments, the frameselection is accomplished by a program implementing one or moreselection criteria.

At Block 310, the media content processing system 103 extractsindividual frames from the selected media content clip. In certainembodiments, the media content clip is exported into individualconsecutive frames, such as 24 to 30 frames per second of playback. Inother embodiments, clips can contain more frames or fewer frames,depending on the format of the source media content.

At Block 315, the media content processing system 103 identifies and/orselects the particular frames that contain a selected character and/orobject and washes the frames through a series of manipulations to removethe selected character from the scene. In certain embodiments, suchmanipulations extend or continue the background image to remove thecharacter and can comprise borrowing pixels from a background where theactor and/or object is not present, retouching the areas with consistentbackground materials, fabricating pixels by filing areas withappropriate artwork from within the frame or other sources and/orblending the areas into the surrounding background.

In certain embodiments, the process is repeated for every play option ineach scene, breaking the clips into multiple video tracks and/or usingediting software to bundle the different tracks into unique“prerecorded” background clips for each option. Within each bundle,different tracks can have unique data file triggers or metadata thatcorrespond to different “in” and/or “out” points within the scene. Forexample, one set of data file triggers can determine when a user imageis to be on or off the screen; another can dictate when a customizedspecial effects layer is activated; a third can command a particularbackground matte layer to appear or disappear as needed.

With audio, a more robust clip development process provides an increasedability to separate audio tracks and/or isolate sound effects, musicalscores, and/or the voices of different characters for individualmanipulation. Media content received from the studios can containmultiple audio tracks separate from the video. For example, tracks 1 and2 can contain mixed audio, while tracks 3 and 4 contain the music and/oreffects. Certain embodiments of a the interactive role performancesystem 100 can either control audio data that has been delivered inseparate tracks and/or mix separate tracks together, or can break audiotracks apart if the source material has them combined. Creating separateaudio tracks allows for the editing of some tracks while not touchingothers. Certain embodiments can substitute and/or remove movie scoreaudio, alter and/or remove actor audio, and/or enhance, alter, and/orremove sound effect data, then later recombine the tracks forassociation with different user play options.

For instance, certain embodiments of the invention can separate theaudio tracks from STAR WARS to remove DARTH VADER'S speaking parts,replace the John Williams score with a royalty-free track, and/orenhance the light saber sound effects. When finished, the system cancondense the separate tracks down to one master track to be played whenthe user chooses to replace DARTH VADER. A similar approach could betaken to alter different tracks for a LUKE SKYWALKER play option. Theresulting experience can have better audio accompaniment because thesound elements can be better manipulated during content development thanthey could be on-the-fly.

At Block 320, the media content processing system 103 creates mattesfrom the media content. In some embodiments, compositing systems involvesuperimposing a new video layer, or “matte,” of the user over theoriginal background content in order to create the illusion that theuser is “in” the prerecorded content. While this effect works well inmany cases, certain prerecorded backgrounds contain foreground elements,such as desks, podiums, castle walls, other actors and/or the like, thatappear in front of the actor to be replaced. In many cases, theseforeground elements also move, such as when a bird flies across theframe, a person walks in front of the actor, and/or a camera moveeffectively changes the position of the stationary wall or desk relativeto the actor in the frame. In order to create a more 3D interactiveexperience, these foreground elements can be recreated or somehow movedso as to be visible in front of the superimposed user's image.

One way for an element to appear in the foreground is by creatingadditional video matte layers during content development. For purposesof this disclosure, mattes can comprise, but are not limited to, videofiles that contain transparency information such that white space allowssubordinate video layers to show through and/or black space preventssubordinate video layers from showing through. Certain mattes can becreated based on elements of the target prerecorded clip such that anyelement which should be “in front” of the user—such as a desk—is black,and/or the elements that should be “behind” the user are white. Thus, incertain embodiments, the matte layers cause portions of a backgroundimage to come to the foreground in front of an inserted user image. Insome embodiments, a moving matte is required for a motion scene. Thematte creation process is described in further detail below. Once amatte is created, it can be synchronized to the media content clip tomatch up with the motion of the object that appears in the foreground.

In certain embodiments, the video recorder 110 captures the user imagewithout making any camera moves, pans or zooms. These functions can beaccomplished through the software of the video composition 120 system.In order to match the original scene, the original scene can be analyzedand metadata 105 can be recorded that captures the in and/or out points,actions in the original scene, audio levels, camera movement, switching,positions, zoom and/or pan. The metadata 105 can further instruct thevideo recorder 110 to scale, move within the x-y coordinates of theoverall combined frame, and/or switch to a different angle. Metadata 105can be recorded in a text document, database, XML file, and/or embeddedwithin the washed content.

At Block 325, the media content processing system 103 records actorposition, size information, and/or other metadata 105 associated withthe washed media content 104. In some embodiments, processing softwarein the media content processing system 103 analyzes the media contentclip to generate metadata, such as the position and size information. Incertain embodiments, the actor position and/or size information are usedduring the setup of the camera, lights and/or green screen to determinethe orientation and/or size of the inserted user in the scene. Usingthis information allows the inserted user image to match as closely aspossible with the character that is being replaced.

At Block 330, the media content processing system 103 creates an outlinegraphic representing the removed character's position in the washedscene. In certain embodiments, the participant uses the outline graphicto determine where he/she should position himself/herself duringrecording of his/her performance of the particular scene. In someembodiments, an outline graphic is not included in the washed scene.Moreover, a user can freely move around within the scene and/or is notrequired to appear in a specific position.

At Block 335, the media processing system 103 transcribes and/orprepares subtitles of the dialogue for each scene or clip. In certainembodiments, subtitles appear when the removed character would bespeaking and disappear when the actor is not. In some embodiments,subtitles may not be required and/or are already available and do notneed to be created.

At Block 340, the media processing system 103 outputs a washed sceneafter completing processing the media content. In certain embodiments,the media processing system saves the washed content into a localstorage device and/or saves the washed content directly to theprerecorded content database 101 as the media content 104. The washedcontent can further undergo a quality control process to ensure that thewashed content has been properly created. The washed content may also besaved into a backup storage system. In certain embodiments, poster artfor display can be created by washing actors out of media content.

FIG. 4A illustrates an alternative embodiment of the media contentpreparation process of FIG. 3. At Block 405, a scene is selected andframes from the scene are created. At Block 410, an actor is removedfrom one frame. At Block 415, a background, such as a wall, is recreatedbehind the actor. At Block 420, the washed frame is extended or repeatedfor the rest of the frames in the scene. In some scenes, the backgroundis similar from one frame to another, and reusing the washed frame savesadditional effort. At Block 425, a track or file with the data triggersfor the in and/or out points of the actor and/or other metadata iscreated. In some embodiments, the in and/or out points are representedby the first and/or last frames the actor appears in. If more than oneactor is selected for removal from the scene, the process can go back toBlock 410 and repeat Blocks 415, 420, and 425 for the next actor. Theprocess can be repeated as many times as necessary for the number ofactors to be washed. At Block 430, one or more tracks with theassociated data triggers are bundled into a single washed media contentscene.

FIG. 4B illustrates another alternative embodiment of the media contentpreparation process of FIG. 3. At Block 450, a scene is selected andframes from the scene are created. At Block 455, elements of the set arereshot and/or a background is digitally recreated either entirely or bycombining the newly shot set elements with the original content at Block460. At Block 465, a track or file with the data triggers and/or othermetadata for the scene is recorded. At Block 470, one or more tracks arebundled into one washed scene.

It will be understood that the media content preparation process can beaccomplished by using any existing or new technologies that can allowfor the altering of video content, such as the ability to map or trackcamera movements from the original content and/or recreate them with analtered background. In addition, any of the described media contentpreparation processes can be used singly or in combination to create thewashed content.

Embodiments of the content development process also allow forcustomization and/or alteration of other elements affecting theinteractive experience. These elements can comprise, but are not limitedto, subtitle data, colors, fonts, placement, actor cues and/orsuggestions, audio and/or video special effects, information about userimage size, location, dynamic movement, color hue, saturation,distortion, play pattern interactivity such as voting, ranking, and/orcommenting, properties for online uploading, sharing, and/or blogging,particulars about creating, sharing, printing movie stills and/orposters based on each scene, gaming elements, pitch, vocals, accuracy,volume, clapping, combinations of the same or the like.

For example, certain analysis can be performed that suggests usersappearing in a scene from LORD OF THE RINGS should appear more orangethan users appearing in a scene from THE MATRIX. Color saturation,lighting, hue data and/or other metadata can be written into the datafiles or metadata 105 for each respective scene, such that during theperformance, the interactive role performance system 100 can use thedata files or metadata 105 to manipulate the live image in order to morerealistically blend the user into the background footage.

Likewise, digital resizing and/or movement data can be programmed intoeach scene that dictates where the user appears in the frame ofprerecorded content, and/or the size of the user image relative to therest of the scene. This information can be used to create dynamiceffects, such as digitally simulating camera movement over the course ofthe scene. This data could also be written into the metadata 105associated with the piece of washed media content 104.

In certain embodiments, the control data or metadata 105 for theseelements is bundled with the associated washed media content 104 and/ormatte layers during content development. These elements can bereferenced and/or controlled with data files which are invisible to theuser, but can be embedded in software elements and/or included indigital files (for example, an Internet downloaded file or XML file) orthe like, appropriately associated with the original content purchasedby the user. These improvements to the content development process canmake the interactive experience more realistic, more immersive, andultimately more enjoyable to the user.

FIGS. 5A to 5D illustrate a frame from a media content during variousphases of certain embodiments of the washing process in which a singleactor is washed out of the scene. For exemplary purposes, the frames, asillustrated, are described hereinafter with reference to the componentsof the interactive role performance system 100 of FIG. 1.

FIG. 5A illustrates a frame from a media content clip processed by themedia content processing system 103. The frame depicts two actors: thefirst actor 505 is the target actor to be washed from the frame whilethe second actor 510 is retained in the frame.

FIG. 5B illustrates the frame of FIG. 5B after the actor 505 has beenwashed from the scene. In certain embodiments, an outline graphic 515 isadded to the washed content to depict the location of the washed actor.The retained actor 510 remains unchanged in the scene. Individual washedframes comprise the complete washed content scenes 104 stored on theprerecorded content database 101.

FIG. 5C illustrates a real-time feed of a user from a video recorder 110superimposed over a washed content, wherein the user image 520 is addedonto the scene. In certain embodiments, the user can use an outlinegraphic to position himself in the scene. That is, the user can moveinto a position such that the user is generally within the position ofthe washed actor as indicated by the outline graphic 515. In someembodiments, the video compositor 120 automatically positions the feedfrom the video recorder 110 in a frame such that an outline graphic isunnecessary by using previously recorded actor position data todetermine where the user image is placed.

FIG. 5D illustrates a frame from a completed combined video. The user520 is inserted into the scene alongside the retained actor 510. Incertain embodiments, the completed combined video is displayed on thedisplay 125. The combined video can also be saved for future playback,or the combined video can be recreated from the washed scene and usercontent without saving the combined video.

FIGS. 6A and 6B illustrate an exemplary matte layer created during themedia content preparation process of FIG. 3. In particular, FIG. 6Aillustrates a matte layer created from the frame illustrated in FIG. 6B.In FIG. 6B, the flight attendant 620 is part of the foreground scene andappears in front of the passenger 630 selected for the washing process.In certain embodiments the matte creation can be performed by “tracing”the particular figure with a digital pointer, frame-by-frame, or usingother software means available to track and/or trace the elements.

The resulting matte layer 610 can be either a moving or stationary videofile used during playback of the washed content to delineate aforeground element of the original source content. Associating thismatte with the real-time user image from the video recorder 110essentially “blocks” the user's image where a foreground object, such asthe flight attendant, covers the user image, and thereby creates theillusion that the user is positioned between background and foregroundelements. By using a moving matte layer, the foreground element can bekept in front of the participant's image even when the foregroundelement moves, such as if the flight attendant moves in front of theuser. The resulting composition advantageously creates a more realistic,multi-dimensional interactive experience.

In yet other embodiments of the invention, additional features can beemployed with that utilize components of the interactive roleperformance system 100 hosted and/or deployed in an online environment.For instance, one method of hosting the content online allows a party oruser to control the storage, filtering, and/or distribution of thefinished video output. In certain embodiments of the technology, a newvideo file is generated with the combined image of the user and theprerecorded content. This “output” file could be saved for laterplayback, shared online, or sold as a DVD to the user in a variety offashions. Though the output of the composition in a single, cohesivevideo stream is relatively efficient, certain problems also can arisewith such an arrangement.

First, in spite of advanced video encryption techniques, users couldfind ways to copy and/or share their recorded files at will. Second,without control over the output content, it can be difficult to policeor filter which output files could be shared online. Third, generatingfully-integrated output files with each user experience can createredundancies in the storage process, increase hosting expenses, and/ordecrease overall system capacity and/or efficiency.

To address these issues, certain systems and methods isolate the user'srecorded performance from the prerecorded background throughout theentire process, such that the images are not combined, except visuallyduring performance playback. In certain embodiments, the washed clip isnot altered or re-recorded during a performance. Rather, the washed clipcan be merely referenced again if a playback option is selected, thenreplayed in tandem with the user's overlay segment.

There are several additional or alternative ways that the video filescan be protected in the disclosed interactive role performance systems.In one embodiment, the prerecorded background content and/or therecorded performance is stored in a non-standard video format such thatit is unplayable in standard video formats or players. In someembodiments, the fact that the images are separate or the backgroundcontent is an individual file is concealed. In certain embodiments, thebackground and/or user media files are stored separately on the user'slocal system.

Other content protection methods can also be used. One method is to lockeach background content clip to a specific operating system, and/orrender them non-transferable between systems. Another method is to makeonly the user file uploadable to a website for hosting and/or sharing,and render the background video unsharable. In some embodiments, anonline system runs an auto query each time an offline system becomes webenabled in order to register the software operating system and/or lockthe content to the that system. Another method is to use a dynamic URLfor a website, and/or change it regularly. In some embodiments, theuploaded clips are digitally “watermarked” in order to track their useshould they be found outside controlled channels. In one embodiment,combined content is stored only on a secure storage location, such as acontrolled server, and only links or references to the protected contentare allowed from programs or applets. The programs can stream the filesfrom the secure storage location without saving a copy of the content.In some embodiments, the programs are authorized by the secure storagelocation before access to the protected content is allowed.

Furthermore, there are processes contemplated in which theuser-generated content can be filtered in order to remove objectionablematerial. One approach is to establish nudity and/or profanity filtersin the finished file upload process. During upload, each performance canbe filtered in real time for nudity and/or profanity, and then assigneda numerical number based on its evaluation. Numbers below a certainbenchmark can be manually reviewed by screeners, and/or numbers below acertain lower benchmark can be automatically rejected and discarded.Another way can be a complete manual review of the user generatedcontent.

One advantage to utilizing the Internet or other network as a platformis the ability to engage multiple users from multiple remote locationswith multiple cameras in numerous forms of interaction.

FIG. 7 illustrates an embodiment of a data flow diagram of aninteractive role performance system configured to operate with multipleplayers in different geographic locations. For instance, a user in NewYork and a user in California can mutually or individually select ascene from STAR WARS to perform, such as with opposite roles. At Block705, the California user selects the scene on his or her interactiverole performance system. At Block 710, the California user selects therole of LUKE SKYWALKER for playing. At Block 720, the New York userselects the same scene on his or her interactive role performancesystem. At Block 725, the New York user chooses the role of DARTH VADER.When the players start the scene and play out the scene, the resultingcomposition is a single ensemble scene, even though the users aregeographically distant. At Block 730, California user data and New Yorkuser data are combined to produce a single ensemble scene, wherein bothparticipant images are combined in the same background scene.

Using an online platform, more complex media bundles and/or data filescan also be quickly accessed and/or executed, making more intricate userexperiences possible. The above multi-player effect, for instance, canrequire the use of additional background content bundles of completelywashed scenes (see above), driven by data files or metadata whichtrigger the camera inputs from each respective user. The multi-camerause could also be executed such that a user in New York selects apreviously performed clip posted by his friend in California, anddecides to act opposite his friend after the fact.

In certain embodiments, this process can require controlled switching ofthe California clip (where the user performed as LUKE SKYWALKER) with awashed content prepared for DARTH VADER in order to constitute thebackground for the new, live user image streaming from New York. Thesemulti-player scenes can thus either be performed live by both parties,or live by one party and prerecorded by the other party. They can alsoeither play the opposite or the same character, and either replacecharacters or simply be inserted into the same scene. In someembodiments, there can be three or more users working together to createa single scene. Multi-camera, multi-location video games can alsofunction well in this environment. It is understood that interactiverole performance system can also be used for multiple players in thesame location (e.g., participants in the same living room).

In certain embodiments, the online environment can be a website forsharing combined video and/or buying additional washed content. Thewebsite allows users to share their combined videos with other viewers.Users can rate videos, allowing videos to be ranked based on popularity.Videos can also be ranked based on number of views, age, and/or otherselection criteria. Users can compete in contests using theirperformances. Users can choose to share videos with select individualsor can choose to make videos publicly available to anyone. Users canalso build social networks with each other.

The website can comprise a home page which displays user informationafter the user logs in. User information can comprise messages,comments, invites, uploads, downloads, viewing statistics, and/orpopularity of performances. The website can further comprise performancegallery pages where combined videos are displayed and where users maysearch for combined videos based on associated metadata. The website canfurther comprise store pages, where additional content may be purchasedfor the interactive role performance system 100. The purchased contentcan then be downloaded to the interactive role performance system 100.

In addition to allowing increased protection, filtering, efficiency,and/or multi-camera playability, the Internet offers several advantages.These comprise, but are not limited to, the ability to generate andmonetize script print-outs, teleprompters and application text outputfor scripts and lyrics, the ability to generate a video introduction tobe used to introduce emails and postings, the ability to select betweenoutput devices including various computer platforms, various multimediaand mobile devices, set-top boxes, and video gaming consoles, theability to download clips with embedded data files, the ability toperform clips with the use of an online interface, the ability to uploadfiles into a sharing forum, vote on clips, share comments, feedback andranking information, and award prizes, the ability to select thesharing/playback information between private/public and limited/massdistribution options, the ability to select between output options andplatforms, the ability to generate still frames and order customizedproducts such as T-shirts containing the generated still frames, theability to utilize 3D rendering and avatar elements to enhance theproduction value, the ability to use video and audio special effectseither before, during, or after a performance, the ability to includeanimation of any kind, the ability to create or utilize video mash-ups,the ability to select additional levels of parental controls and contentfiltering, the ability to manipulate content through audio and videomixing tools, editing suites, mash-up controls, and the like, and/or theability to record new content such as audio information to mix into theclips.

In certain embodiments, the interactive role performance system 100provides a user interface for the user to control the video compositingprocess. FIG. 8 illustrates an embodiment of a wireframe 800 of variouspages of a video compositing interface.

In certain embodiments, the interactive role performance system 100provides a graphical user interface for the user to view and/or selectwashed scenes and/or combined video scenes. A cascade user interface canadvantageously allow the user to view a plurality of scenes or datatiles on one screen (Block 805). In some embodiments, the cascadeinterface comprises a plurality of rows and columns of images of scenes.The scenes can be still or static images and/or video clips. FIG. 9illustrates an exemplary screen display of one embodiment of the cascadeinterface.

As illustrated in FIG. 9, the display 900 includes four columns and fiverows of screen or data tiles arranged in a three dimensional array. Eachof the tiles further includes a graphical representation of the mediacontent that it represents, such as still images of movies. Theillustrated bottom, front or first row position 905, displays the“closest” scenes or screen tiles to the user. Close scenes can bedenoted by a color image, unless the scene is from a black and whitemovie, larger size, and/or a title. Scenes on “farther” rows areprogressively grayed out and/or smaller. The “closer” scenes partiallyoverlay the subsequent “farther” scenes. Additional information can besuperimposed on the image, such as the number of washed scenes, therun-time of scenes, the number of combined video created using washedscenes from the movie 915, and/or the like. Scene ordering can becontextual based. For example, the most recently selected scenes canappear on the first row position 905, with less used scenes displayed onprogressively further rows.

In FIG. 9, the interface is “focused” on the first row of data tiles,that is, the selected scene is one from the first row. Keystrokes orother user controls can send a selection command to the interface thatcan move the focus from one selected scene to another on the first row.Focus can be shifted to another row by moving the cascade and selectinga new first row and/or by using a mouse to select a clip on another row.

In FIG. 9, up to 20 scenes can be displayed at one time. Other scenesare displayed by “rolling” or shifting the cascade. For example, thefirst row position 905 consists of scenes 1-4, the second row position907 consists of 5-8 and so on until the fifth row position 913 of 17-20.Scenes above 20 are undisplayed. The user can use an input device, suchas a keyboard, keypad, touch screen, mouse, remote, and/or the like tosend a navigation command to the interface to roll down the cascade. Thefirst row of data tiles can be rolled or shifted out of the currentselection with the second row of scenes 5-8 appearing in the first orfront row position 905. Subsequent rows move to “closer” row positions.A new fifth row with scenes 21-24 appears in the furthest, end or backrow position 913.

In certain embodiments, the cascade can be rolled until the undisplayedscenes are sequentially displayed to the user. The cascade can stoprolling once the last scene is displayed or it can loop back to theinitial first row, with scenes 1-4 appearing in the fifth row position913 with the user able to keep rolling the cascade and repeating thedisplay of the scenes. The cascade can also be rolled up, with newscenes appearing as the closest row 905 instead of the farthest or endrow 913. It is understood that fewer or greater number of scenes can bedisplayed by using fewer or greater numbers of rows and/or columns. Incertain embodiments, more than four columns can be displayed. In someembodiments, less than four columns can be displayed. In certainembodiments, more than five rows can be displayed. In some embodiments,fewer than five rows can be displayed. The number of rows and columnsused can depend on the number of scenes to be displayed on a singlescreen.

In FIG. 9, filters can further be applied to the scenes such that onlycertain categories of scenes are displayed. In certain embodiments,selectable filters 930, 935 are displayed at the top of the cascadeinterface. Scenes can be filtered based on categories such as availablemovie clips, movie content ratings (e.g., “G,” “PG,” “R,” etc.), and/orperformances of combined videos. Scenes can also be filtered based oncategories such as movies, TV, commercials sports, emotives,combinations of the above, or the like. A search bar can also allow theuser to search for specific scenes. Searches can be based on actors,move titles, scene names, descriptions, and/or the like.

FIG. 10 illustrates an exemplary screen display of one embodiment of themovement and selection process of the cascade interface of FIG. 9. Theuser can roll down the cascade, causing new images to be displayed. Incertain embodiments, when the user holds down a button on the mousewhile the mouse pointer is over the cascade, the mouse pointer changesto a gripping hand, indicating the user has grabbed the cascade and cannow roll the cascade. Dragging up can roll the cascade up, whiledragging down can roll the cascade down. The cascade can roll throughmultiple rows depending on how far the user moves the mouse. After theuser finishes rolling the cascade, the displayed scenes appear in thenormal cascade configuration of FIG. 9. The user can then select animage. It is understood that other input devices can be used to controlthe cascade, including, but not limited to, a keyboard, arrow keys, amouse, a remote control, a touch pad, or the like.

A selected image can display a play icon so that the user can play thescene corresponding to the image. The select screen of FIG. 10illustrates one embodiment where selection converts the image to a videoclip so that the movie scene is played in the cascade. In someembodiments, hovering a cursor over the scene can cause the scene toautomatically play. Selecting a scene can also cause the cascadeinterface to proceed to another screen, such as the performance screenat Block 810 in FIG. 8, which displays the washed content from themovie, and the selectable actors. Selecting a row and/or clip can alsocause the cascade to “fold down” into a single row, with the furtherrows being folded into the first row simulating a stack of cards or aROLODEX.

It will be recognized that the cascade can operate in various mannersand is not limited to displaying scenes. For example, the cascade coulddisplay the closest images in the top row instead of the bottom row. Thecascade could be rolled horizontally instead of vertically. The usercould use a keyboard, touch screen, keypad and/or remote to move thecascade. The user could select the number of rows and columns that makeup the cascade. The user could re-order the images by moving images intodifferent positions. Closer and farther images could be indicated usingother visual cues or no cues could be used. The cascade could be used todisplay titles, DVD covers, album covers, photographs, icons, and/orother images.

Referring back to FIG. 8, after an image in the cascade is selected, thewireframe 800 moves to Block 810, where a scene to be performed can beselected. FIG. 11 illustrates an exemplary screen display of oneembodiment of the performance screen. A cascade interface 1105 displaysthe available washed content from the selected movie. The cascadeinterface 1105 of FIG. 11 can behave similarly to the cascade interfacesof FIGS. 9 and 10. A large display window 1110 can display the washedcontent scene in a higher resolution. Scene information 1115 associatedwith the washed content can also be displayed, and may comprise, forexample, editable title, description, category, and/or tags associatedwith the washed content.

After a performance is selected, the wireframe 800 proceeds to a roleselection screen at Block 815. The role selection screen allows a userto select an actor to play, to be added in the scene as an extra, and/orto select a completely washed clip where no actors are left. FIG. 12illustrates one embodiment of the role selection screen.

Moreover, in certain embodiments, the user can choose to display alarger view of the display window when viewing a scene. FIG. 13illustrates one embodiment of a large screen view of the display window.

After selecting a role, the user can print out a script of the lines inthe scene. FIG. 14 illustrates an exemplary screen display of oneembodiment of a script printing screen. In certain embodiments, thescript can be provided as a PDF, text, Word document, image, and/orother file type.

Referring back to FIG. 8, at Block 820, the user is instructed on how tosetup the camera. FIG. 15 illustrates an exemplary screen display of oneembodiment of the camera setup screen. The instructions can comprisepositioning information of the user relative to the camera, green screenand/or monitor. Before recording can proceed, the camera can capture areference frame of the scene. FIG. 16 illustrates an exemplary screendisplay of one embodiment of the reference frame setup. The user can beinstructed to step out of the scene and press record to capture areference frame of the background. In certain embodiments, the cameraauto focus and/or white exposure may be turned off to get a moreconsistent background image.

At Block 825, the wireframe 800 moves to a record video screen, whereinthe participant records a video of himself or herself to be combinedwith the washed content. For instance, the video combining process canbe include the compositing process 200 described above with reference toFIG. 2. In some embodiments, another role can be selected, allowing oneparticipant to play multiple roles or more than one participant to playroles in the same scene.

At Block 830, the user can add an introduction for the combined video.FIG. 17 illustrates an exemplary screen display of one embodiment of anadd introduction screen. In certain embodiments, a cascade displaysavailable backgrounds. The background can be a message, advertisement,product placement, logo, still image, combinations of the above, or thelike. A display window shows a larger image of the selected background.The user can record an introduction using a process similar to the videocompositing process 200 described in FIG. 2. The user can add metadatato the introduction, such as title, description, category, and/or tags.Once the combined video is complete, the user can upload the video to acentral storage for sharing, such as a website.

The user can access the settings screen, at Block 835, from many of theinterface screens. FIGS. 18-20 illustrate exemplary screen displays ofone embodiment of the setting screens. The user can determine recordingsettings, user account settings, and/or parental control settings.

It should be noted that the above developments would accompany anyembodiment of the system, whether as a stand-alone hardware device forthe living room, a computer-based system of any platform, on video gamesystems of any video game platform, any mobile technology, any publicvenue system or kiosk, or any other foreseeable embodiment.

In certain embodiments, the interactive role performance system 100 canbe provided in a self-contained, mobile unit. The mobile unit can be amovable kiosk, an automobile, and/or a portable device. The mobile unitscan be set up at college campuses, high schools, movie theaters,retailers and/or other public venues. Users can use the mobile units tocreate performances without having to purchase their own system.

In some embodiments, the interactive role performance system 100 isprovided in a mobile device, such as a laptop, PDA, cell phone, smartphone, or the like. The mobile device can be used to view, preview,and/or record media content. In some embodiments, the mobile device isconnected to an online content database from which the mobile device canupload participant performances and download washed content and otherusers' performances.

In certain embodiments, the interactive role performance system 100 canbe provided as a package comprising of a green screen, a stand for thescreen, a USB camera, a camera hook or clip, a remote, a tripod, and/ora CD or DVD containing software implementing the functions of theinteractive role performance system and a number of prerecorded content.Moreover, systems and methods disclosed herein can be advantageouslyused with the video compositing systems and method disclosed in U.S.Pat. No. 7,528,890, issued May 5, 2009, which is hereby incorporatedherein by reference to be considered part of this specification.

In some embodiments, the interactive role performance system 100 can beused in a gaming system. For example, a gamer can use the interactiverole performance system 100 to record his actions and insert it into agame. The game could be a music video game where the gamer is playing amusical instrument. The gamer's image could be recorded and insertedinto the game as a band member playing a song onstage. The gamer couldalso be inserted into a music video for the song that the gamer isplaying. The interactive role performance system 100 can be used inother types of games, such as a movie making game, a fighting game,and/or a role playing game.

Similarly, the system can be used in a variety of markets ordistribution channels, such as education, airlines, prisons, or forgaming, dating, corporate training, education, professional services,and/or entertainment use, in either the U.S. or internationally. It canbe used for advertising or promotions, product placement, viralmarketing, on-line sharing, contests, surveys, consumer products,affiliate programs, clothing and apparel, still photographs, avatars,greeting cards, mash-ups, hardware, software, or licensing.

The content may be, but is not limited to film, television, music, musicvideos, documentaries, news, sports, video games, original content,user-generated content, licensed content, royalty free content, anypre-existing moving image or graphic content, still images, digitalavatars, and/or online content. For example, a user can replace a sportscommentator in a sports clip and provide alternate commentary, givinghis own analysis and/or opinion of the game. The content may or may notinclude audio, dialogue, and/or effects. The content can be in Englishor any other language.

The user experience might include, but would not be limited to, akeyboard, mouse, manual, or remote user interface, the use of a wired orwireless webcam, camera positioning via manual or digital means, soundrecording by means of one or more wired, wireless, or built-inmicrophones, accessories such as props, costumes, a colored green screenwith or without a stand, no green screen, coin-operated kiosks with orwithout an operator or operators, automated interface navigation withmanual or automatic data entry, automated demos, tutorials, andexplanations, any type of compositing—with or without a chroma key,and/or any type of output on any platform.

Furthermore, in certain embodiments, the systems and methods describedherein can advantageously be implemented using computer software,hardware, firmware, or any combination of software, hardware, andfirmware. In one embodiment, the system is implemented as a number ofsoftware modules that comprise computer executable code for performingthe functions described herein. In certain embodiments, thecomputer-executable code is executed on one or more general purposecomputers. However, a skilled artisan will appreciate, in light of thisdisclosure, that any module that can be implemented using software to beexecuted on a general purpose computer can also be implemented using adifferent combination of hardware, software or firmware. For example,such a module can be implemented completely in hardware using acombination of integrated circuits. Alternatively or additionally, sucha module can be implemented completely or partially using specializedcomputers designed to perform the particular functions described hereinrather than by general purpose computers.

Moreover, certain embodiments of the invention are described withreference to methods, apparatus (systems) and computer program productsthat can be implemented by computer program instructions. These computerprogram instructions can be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing the actsspecified herein to transform data from a first state to a second state.

These computer program instructions can be stored in a computer-readablememory that can direct a computer or other programmable data processingapparatus to operate in a particular manner, such that the instructionsstored in the computer-readable memory produce an article of manufactureincluding instruction means which implement the acts specified herein.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing the acts specified herein.

While certain embodiments of the inventions have been described, theseembodiments have been presented by way of example only, and are notintended to limit the scope of the disclosure. Indeed, the novel methodsand systems described herein may be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods and systems described herein may be made withoutdeparting from the spirit of the disclosure. The accompanying claims andtheir equivalents are intended to cover such forms or modifications aswould fall within the scope and spirit of the disclosure.

What is claimed is:
 1. A method for combining video signals in aninteractive video system, the method comprising: receiving a first videosignal associated with pre-washed video content, the pre-washed videocontent having been created from prerecorded video content havingmultiple frames by digitally removing an original character therefrom inselected ones of the multiple frames; receiving metadata associated withthe first video signal, the metadata being indicative of the selectedframes; receiving a second video signal in real-time comprising secondvideo content having a background image and a video image of aparticipant playing a role of the original character in front of thebackground image; filtering the second video signal in real-time toremove the background image to create a modified second video signalwith the video image of the participant; and overlaying in real-time thefirst video signal and, as determined by the metadata, the modifiedsecond video signal corresponding to the selected frames to create acombined video signal, whereby the video image of the participant in theselected frames appears in place of the original character of theprerecorded video content.
 2. The method of claim 1, additionallycomprising displaying the combined video signal in real time withrespect to said receiving the second video signal.
 3. The method ofclaim 1, wherein said filtering comprises: receiving a third signalconsisting essentially of the background image; comparing the secondvideo signal with the third signal to identify matching portions of thesecond video signal and the third signal; and modifying the second videosignal by designating the matching portions within the second videosignal to be transparent.
 4. The method of claim 1, wherein theprerecorded video content comprises a portion of at least one of amovie, a television show, and a commercial.
 5. The method of claim 1,wherein the prerecorded video content comprises a still image.
 6. Themethod of claim 1, wherein said receiving the first video signalcomprises receiving the first video signal via a network.
 7. The methodof claim 1, additionally comprising displaying a script to theparticipant during said receiving the second video signal, wherein thescript corresponds to lines of the original character.
 8. The method ofclaim 1, additionally comprising storing the modified second videosignal separate from the first video signal for later overlaying anddisplaying.
 9. The method of claim 8, wherein said storing comprisesuploading the modified second video signal onto a network.
 10. Themethod of claim 1, wherein in the pre-washed video content, a secondoriginal character has been digitally removed from the prerecorded videocontent in a second selected ones of the multiple frames, additionallycomprising: receiving second metadata indicative of the second selectedframes; receiving a third video signal comprising a second video imageof a second participant; and overlaying the first video signal and, asdetermined by the second metadata, the third video signal correspondingto the second selected frames, whereby the second video image of thesecond participant in the second selected frames appears in place of thesecond original character of the prerecorded video content.
 11. Themethod of claim 1, wherein said overlaying further comprises placing oneor more elements of the prerecorded video content delineated by a mattelayer in a foreground of the combined video content to overlay at leasta portion of the video image of the participant.
 12. The method of claim1, wherein in the selected frames of the pre-washed video content, theoriginal character has been digitally replaced with the backgroundimagery surrounding the original character in the prerecorded videocontent.
 13. The method of claim 1, wherein the participant steps awayfrom a video camera to enable the video camera to record a referencebackground image, and the participant then steps back in for generatingthe second video signal.
 14. The method of claim 13, wherein, prior tothe participant stepping away from the video camera, the video camerarecords a reference image of the participant in front of the backgroundimage.
 15. The method of claim 13, wherein the filtering step includessetting each pixel of the background image as transparent or opaque,based on a color channel difference between the second video signal andthe reference background image for the pixel.
 16. The method of claim 1,wherein the filtering step includes setting each pixel of the backgroundimage as transparent or opaque, based on a color channel differencebetween the second video signal and a reference background image for thepixel.
 17. The method of claim 16, wherein the participant steps awayfrom a video camera to permit the video camera to record the referencebackground image.
 18. A system for interactive video entertainment, thesystem comprising: a first database comprising, pre-washed video contenthaving been created from prerecorded video content having multipleframes by digitally removing an original character therefrom in selectedones of the multiple frames, and metadata associated with the pre-washedvideo content, the metadata identifying the selected frames; a videoinput configured to receive a real-time video signal comprising a videoimage of a participant and a background image; and a processing moduleconfigured to execute on a computing device, the processing moduleconfigured to, filter in real-time the real-time video signal to removethe background image to create a modified real-time video signal, andoverlay in real-time, based at least on the metadata, a first videosignal created from the pre-washed video content and the modifiedreal-time video signal corresponding to the selected frames to createcombined video signal, whereby the video image of the participant in theselected frames appears in place of the original character of theprerecorded video content.
 19. The system of claim 18, furthercomprising a video recorder configured to generate the real-time videosignal and to transmit the real-time video signal to the video input.20. The system of claim 19, wherein the video recorder comprises awebcam.
 21. The system of claim 18, wherein the processing module isfurther configured to cause a display of the combined video signal. 22.The system of claim 18, wherein the processing module is furtherconfigured to cause a display of a character outline at a location wherethe original character was digitally removed from the prerecorded videocontent.
 23. The system of claim 18, further comprising a seconddatabase configured to store second video content based only on themodified real-time video signal.
 24. The system of claim 18, wherein theprocessing module comprises a graphics processing unit (GPU).
 25. Thesystem of claim 18, wherein in the selected frames of the pre-washedvideo content, the original character has been digitally replaced withthe background imagery surrounding the original character in theprerecorded video content.
 26. A system for combining video signals, thesystem comprising: a first receiver for receiving a first video signalassociated with pre-washed video content, the pre-washed video contenthaving been created from prerecorded video content having multipleframes by digitally removing an original character therefrom in selectedones of the multiple frames, and for receiving metadata associated withthe first video signal, the metadata being indicative of the selectedframes; a second receiver for receiving a second video signal inreal-time comprising a video image of a participant and a backgroundimage; a filter for filtering the second video signal in real-time toremove the background image and to create a modified second videosignal; and a video compositor for overlaying in real-time, based atleast on said metadata, the first video signal and the modified secondvideo signal corresponding to the selected frames to create a combinedvideo signal, whereby the video image of the participant in the selectedframes appears in place of the original character of the prerecordedvideo content.
 27. The system of claim 26, further comprising a displayfor displaying in real time the combined video signal.
 28. The system ofclaim 26, wherein in the selected frames of the pre-washed videocontent, the original character has been digitally replaced with thebackground imagery surrounding the original character in the prerecordedvideo content.
 29. The system of claim 26, wherein the participant stepsaway from a video camera to permit the video camera to record areference background image, and then the participant steps back in forgenerating the second video signal.
 30. The system of claim 29, wherein,prior to the participant stepping away from the video camera, the videocamera records a reference image of the participant in front of thebackground image.
 31. The system of claim 29, wherein the filteringincludes setting each pixel of the background image as transparent oropaque, based on a color channel difference between the second videosignal and the reference background image for the pixel.
 32. The systemof claim 26, wherein the filtering includes setting each pixel of thebackground image as transparent or opaque, based on a color channeldifference between the second video signal and a reference backgroundimage for the pixel.
 33. The system of claim 32, wherein the participantsteps away from a video camera to permit the video camera to record thereference background image.
 34. The system of claim 26, wherein saidoverlaying further comprises placing one or more elements of theprerecorded video content delineated by a matte layer in a foreground ofthe combined video content to overlay at least a portion of the videoimage of the participant.